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This report provides a comparative cost analysis of 
three different methods of developing final examinations for 
' secondary schools in the Montgomery County Public Schools, Maryland. 
Based on expenditure, reports and interviews with central and 
school-based staff, cost estimates were developed for three kinds of 
exams: (1) the county-wide final exam/departmental exam approach 
mandated by Senior High School Policy; (2) departmental final exams 
alone; and (3) final exams made by individual teachers. The study 
found that the Board-mandated approach is by far the most expensive 
of the three alternatives, costing from 30 to 150 percent more than 
the other approaches^ It should be noted that over 95 percent of the 
costs which were recorded in this analysis were not incremental or 
out-of-pocket costs but rather were "opportunity costs" resulting 
from (1) providing teachers with a half day of release time each 
semester^ to work with their colleagues on jointly developed 
departmental exams, (2) teachers spending additional time developing 
final exams, and (3) allocating roughly 25 percent of academic 
subject /coordinator time and 1150 equivalent days during the previous 
summer to the development of uniform countywide exams. (Author/BS) 
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SUMMARY 



This report provides a .comparative cost analysis of three different methods of 
developing final examinations for secondary schools in MCPS. Based on 
expenditure reports and interviews with central and sch(»l-based staff, cost 
estimates were developed for three kinds of final exams: the countywide final 
exam/departmental exam approach mandated by the Senior High School Policy, 
departmental final exams alone, and final exams made by individual teachers. 
The study found that the Board-mandated approach' is by far the most expensive 
of the three alternatives, costing from 30 to 150 percent more than the other 
approaches . 

However, it should also be noted that over 95 percent of the costs which were 
recorded in this analysis were not incremental or out-of-pocket costs but 
.rather were "opportunity costs" resulting from a) providing teachers with a 
half day of release .time each semester to work with their colleagues on 
jointly developed departmental exams, b) teachers spending additional time 
developing final exams, and c) ..allocating roughly 25 percent of academic 
subject -coordinator tine and 1150 EYE days last summer to the development of 
uniform countywide exams. 



'BACKGROUND 

On February 12, 1980, the Montgomery County Public Schools (MCPS) Board ' of 
Education adopted a policy statement on The Senior Higfi School (Resolution 
No. 161-80). In addition to providing staffing guidelines\ regarding MCPS's 
resvonsibilities in educating high school students, the B^oard.of Education 
specifically provided for the development and administration of two kinds of 
final examinations in all major high school subjects: school-based 
departmental examinations and uniform countywide examinations. \ In addition, 
because of the many questions which were- raised about these uniform countywide 
examinations, the policy established a three-year pilot proje.ct \iuring which 
the new examination procedures would be developed, refined, and evaluated for' 
English and mathematics. At- present, this pjLlot is in its third \and final 
year of operation. ' "^s^ 

In December, 1982, the newly elected Board of Education asked the Department 
of Educational Accountability (DEA) to conduct a preliminary cost analysis of 
the countywide examination process to date and to present the results 'at the 
February 8, 1983, Board " meeting. It was acknowledged at that tj^me \ that 
because of the limited time frame only a limited amount of data could be 
collected, .and analyses would have, to be based largely on reports \ of 
mathematics staff involved in the administration of a math exam^ in June, 1982, 
and English staff preparing for the administration of the pilot 'English exams 
in January, 1983. (See the Technical Appendix for a description of the data, 
collection procedures.) - . ^ ' 
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STUDY QUESTIONS 



The basic question v ^he study was designed to address were: 

o What ar.c ^.osts associated with the development and 

administrate / iform countywlde examinations? 

o Kow do these ' compare with those of other types of examinations 

such as departmenu^- . exams or teacher made exams? 

o What are the Tong-range implications of continuing to develop 
uniform countyw±d« examinations ip their current form? 

' ]■• 

To. address thc.-je Issvhk data were gathered on the following cost components 
for each of the categories jf exams listed above: 

o Teat development 

o Preparation for teat administration 
o Materials, 
o Scoring 

These data were then used to estimate total costs for current exams in 
Geometry B, Introduction to High School English, and- Narrative-Drama as 
Literature I as well as" to project the costs of extending the pilot to all 
secondary English and mathematics courses. The appendix presents greater 
detail on the costing methodology which was used, 

FINDINGS 

The study's findings are divided into three sections: what was learned from 
the study about the estimated costs" of administering the final examinations in 
English and mathematics, the, relative costs of each. of the components of the 
alternative testing procedures, and cost projections for a full-blown final 
exam program in English and mathematics based on these data. 

Estimated Costs for Pilot Exams 

Tables 1 and 2 present the findings on- costs for the pilot programs in 
Geometry B, Introduction to ' High School English, and Narrative-Drama as ■ 
Literature I (see columns- entitled "Uniform. Portion" and "Depti Portion"). In 
addition, costs, for exams in- other English, and mathematics courses not 
included in the" countywide pilot program are also presented in the columns 
entitled "Other Dept. Exam" and - "Teacher-made Exam." The tables show the 
following: - - 

o Under the pilot system, the estimatjed costs for a " single countywide 
final exam were $57,550 in English and $34,400 in mathematics. 

o Comparison of these fijgures with those for teacher-made exams or 
departmental exams alone shows that the current plan is clearly the 
most costly. In English, the pilot program is approximately 150 
percent more cpstly than teacher-made exams ($57,650 vs.. $23,800) and 
30 percent more costly than departmental exams alone ($57,650 vs. 
$44,426). In mathematics, the pilot program -is almost 50 percent 
_ more costly than teacher-made exams ($34,400 vs. $23,309) and 33 
percent most costly than departmental exams ($34,400 vs. $25,858). 



TABLE 1 



Total Costs of Three Type of Final Exams in English* 

Total Cost 

Total -Countywide Exam . $57,550 

Uniform Portion 11,693 

Dept. Portion** 45,857 

Other Dept. Exam** 44,426 

Teacher-made Exam*** 23,800 

*Based on an enrollment of 3,400 students, the actual enrollment of 
Introduction ' to High School English and Narrative-Drama as Literature I in the 
fall of 19iB2. ^ '^^ 

**The slight difference in the coats of the "departmental portion" of the 
countywide exam and - the "other departmental exams" is probably accounted for 
by the extra time needed to "wrap" the departmental exam around the countywide 
portion. 

***This assumes there is an average of two teachers per school teaching a 
course. 



TABLE 2 

Total Costs "of Three Types of Final Exams in Mathematics* 

Total Cost 

Total Countywide Exam $34,400 

Uniform Portion . 7,298 

Dept. Portion** 27,102 • 

Other Dept. Exam** 25,858 
Teacher-made Exam*** -23,309 

*Based on an enrollment of 5,600 students, the actual enrollment of Geometry 
B in the spring of 1982. 

**The slight difference in . the costs of the "departmental portion" of the 
countywide exam and the "other departmental exams" is probably accounted for 
by the extra time needed to "wrap^^* the departmental exam around the countywide 
portion. 

**This assumes there is an average of two teachers per school teaching a 
.course. ' . 
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Component Costs 



Tables 3 and 4 show the relative costs of each of the four components of the 
testing procedure examined: development, preparation, materials, and 
scoring.. The data show the following: 

o Generally, the most costly part of the pilot program was test 
development, accounting for 72 percent of the cost - of the pilot 
program in English jand 61 percent of the pilot program in 
mathematics. When the other approaches to final exams are 
considered, .development remains the most costly item. Only for 
teacher-made examinations in mathematics does this pattern change. 
Here, only 28 percent of the cost is attributable to development. 
This may well be because prototypes for such tests readily accompany 
' most mathematics texts as well as the MCPS curricular guides. 

o Large differences are found in the costs of scoring. Costs for 
scoring are proportionally lower for the pilot program than the other 
alternatives. In English, the percentage of total costs related to 
scoring is 16 percent for the pilot program, compared to 28 percent 
for the other departmental exams and 24 percent for the teacher-made 
exams. In mathematics, the comparable figures are 21 percent for the 
pilot program, compared to '30 percent for other departmental exams 
and 62 percent for teacher-made exams. The apparent cost savings in 
the pilot program are largely attributable to the use of multiple 
choice machine-scoreable answer sheets (Scantron) for the uniform 
countywide portion of the tests and for some of the tests used for 
the departmental portions. 

These findings suggest that many students are. given tests in these three 
courses consisting mostly of multiple choice questions. When asked' why the 
departmental portion of many of the pilot program exams were mostly multiple' 
choice ^ items, the., most common response was that the short timeline between 
administering these tests and submitting grades precluded the use of many 
lessays, proofs, and other forms of constructed responses. 
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TABLE 3 



Relative Costs of Development, Preparation, 
Materials, and Scoring for English* 





Development 


Preparation 


Materials 


Scoring 




% 


% 


% 


Z 


f 

Total Countywide Exam 


7.2 


12 


1 


16 


Dnif orm Portion 


71 


• 10. 


3 


16 


Dept. Portion 


72 


12 


.4 


16 


Other Dept. Exam 


62 


10 


1 


28 


Teacher-made Exam** 


66 


9 


1 


24 


♦Assumes an enrollment 


of 3,400 


students, the 


actual enrollments 



Introduction to High School English and Narrative-Drama as -Literature I in the 
fall. of 1982. 

**This assumes there is an average of two teachers per school teaching a 
course. 



TABLE 4 

Relative Costs of Development, Preparation, 
Materials, and Scoring For Mathematics* 





Development 


Preparation 


Materials 


Scoring 




% 


% 


Z 


Z 


Total Countywide Exam 


61 . 


16 


2 


21 


Uniform Portion 


61 


23 


7 


10 


Dept. Portion 


. 61. 


14 


1 


24 


Other Dept. Exam 


54 


15 


1 


.30 


Teacher-made Exam** 


28 


,9 


1 


62 



^Assumes an enrollment of 5,600 students, the actual enrollment of Geometry B 
in -the spring, of 1982. • ' ■ ■ : 

**This assumes there is., an average of two teachers per school teaching a 
course. 



PROJECTED COSTS 



Using the data obtained above, coats were projected for implementing ifinal 
examinations in all courses for English and mathematics. Tables 5 and 6 
present these calculations. The tables show that for a single semester the 
' ■ P'^°'^"i'^8 uniform countywide and departmental examinations would be 

^ ^,Oi^ ^TOI.OIO for all English courses and $339j977 for all mathematics courses (or 
a total of $1,106,059 per semester). If we compare these figures to the 
cost of teacher-made exams, we find that the current plan costs approximately 
250 percent as much as would one involving teacher-made exams alone. If we 
did not have uniform countywide exams but relied solely on'departmental exajns, 
we would still find such a system about 70 percent more costly than using 
teacher-made exams alone. 

In examining the cost figures, it must be pointed out that despite the 
apparent costliness of the pilot . program the ^numbers presented may well be 
underestimates of what a full-fledged program designed to fulfill the intent 
of Senior -High Policy would cost. That is, if th*i pilot program is to provide 
. data for program monitoring, the test data produced by each school should be 
aggregated and analyzed so that instances in which scores are particularly low 
can be thoroughly explored. The program at present is too -new to know a) 
exactly how this might best be done and b) how much such activities will add 
to the estimates provided in this paper. Preliminary explorations of this 
question -suggest, however, that some complex technical and conceptual problems 
will have to be resolved. 



1. It should be noted that the cost of the pilot program is not greatly 
affected by either the actual enrollment in courses or fluctuations in 
enrollment because of the relatively large portion of the cost attributable to 
development (which is not affected by . course enrollments) and the small 
proportion attributable to materials and scoring. 
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TABLE 5- 

Projected Total Cost of Three. Types of Final Examinations in English* 



Total Cost 

Total Countywlde Exam $766,082 

Uniform Portion 307 ,648 

Dept. Portion 458,434 
Other Dept. Exam 442,846 
Teacher-made Exam** 292,974 

*This assumes 30 courses offered countywide and 10 courses offered per 
school. . o . 

**This assumes two teachers per course in each school. 



TABLE 6 

Projected Total Cost of Three Types of Final Exams in Mathematics* 



Total Countywide Exam 
Uniform Portion 
Dept. Portion 
Other Dept. Exam 
Teacher-made Exam** 



103,555 
236,422 



Total Cost 



$339,977 



218,356 
159,063 



*This assumes 16 courses offered countywide and 10 courses offered' per 
school. . 



*'^This assumes two teachers per course in each school. 



ADDITIONAL CONSIDEBATIONS 



In examining the cost figures presented above, it must be pointed out that in 
most cases they do not represent incremental or out-of-pocket costs, but 
rather opportunity costs. That is, if the teachers were not preparing exams 
they would be doing something else with that time. Only the niaterials costs, 
ranging from 1 to 7 percent, represent true additional costs to the^ school 
system. Therefore, one should not be mislead into thinking that elimination 
of one or more parts of the exam process ^uld result in substantial actual 
dollar savings to the school system. Rather, the real question is whether or 
not it is cost effective to spend MCPS resources as the system is now doing, 
or whether the resources would be better used in some other way. 

In addition, the data that have been presented here provide a very 
unidimensional look at* a multidimensional ^picture. They do not tell whether 
the current plan is having atiy effect on the consistency of instruction across 
or within, different schools nor whether the information provided by the pilot 
program is either used or useful for program monitoring. Comments received 
from respondents showed that participating staff themselves* had differing 
views of the value of the program. Some felt that the exams definitely were 
useful and praised their quality. In fact, they expressed the desire to have 
access to the test items for their own use. Others felt the uniform * 
countywide exams were too easy and did not measure what the students had 
learned. 

Finally, far too little is known about the relati.ve strengths sLd weaknesses 
of approaches other, than the current pilot program which could also promote 
program consistency across schools. There may well be some alternative ways 
in which departmental or teacher-made exams could be used for this purpose. 
For example, uniformity could be assessed by analyzing the content ' and 
coverage of objectives in final exams in a limited number of courses each 
year. Also, -item pools covering selected objectives might be made available 
to be incorporated into locally generated tests, with performance on these 
items compared across schools. Clearly, neither of • these alteratives would . 
eliminate the costs of test development- or get around the problem of 
allocating resources to the analysis of the exams and the data produced. They 
might well cut down, however, on the spiraling costs that have been documented 
here. . ■ ' 
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TECHNICAL APPENDIX 



This appendix provides some background information as to how the analyses of 
exam costs werje organized and carried out. The first section describes the 
types of exams that were compared in the report. The second section lists the 
various kinds of data that were used. The third section discusses the data 
sources. The final sections provide details, on how the data were summarized. 

Types of Exams 

Three types of final exams were studied. 

1* The countywide exam is divided into a uniform countywide portion and a 
«school departmental portion. 

a. The uniform countywide portion - is the half developed by the 
Department of Academic Skills and is administered in all schools. 

b. The departmental portion is the half developed by each school and is 
unique to each school. ? 

2. Other departmental exams are developed by each department in each school 
for courses that are not part of the countywide exam pilot. 

3. Teacher-made final exams are those developed by an individual teacher. 
Because all finals in major subjects must be. departmentalized, these exams 
represent exams written for subjects in a school for which there is only 
one teacher teaching all of the course's sections. 

Categories of Cost Estimates 

Costs were divided into four components. The factors considered in each are 
described below. 

1. Development 

a. School costs— teacher and/or resource teacher time to write the 
exam, meeting time to discuss how -the writing will be done, time to 
review the final product". 

b. County costs~EYE teacher, . teacher specialist, coordinator, and 
consultant time to write the items for each test. ^ # / ^ 

2. Preparation 

a. School costs — teacher, ' resource teacher, secretary, and/or aide time 
to. type, proof, run off, and collate, the exams; time to prepare 
materials for review that were necessary beyond what teachers would 
normally do for an exam period; time to get' all materials (tests, 
pencils, paper-, answer sheets, etc.) ready for te'achers to administer 
.the test., T ' . 
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b. County costs — typists and clerical time for aiding the item writers; 
^ ' labor involved in printing and distributing the tests. . 

Materials 

a. School costs — paper for printing tests, scrap paper, answer .sheets 

b. . County costs—workshop materials, materials involved in printing, 

answer sheets ** . / 

4. Scoring ^ 

a. School costs — teacher, resource teacher, and/or aide time to score 
tests either on a scanning machine or by hand; time to analyze 
results. 

b. No county scoring, yas done. 
Sources of Data 

The information used to make the cost estimates came from a variety of 
sources. Development and part of" the preparation and materials costs for the 
uniform countywlde portion were obtained from subject coordinators ±xt. the 
Division of Academic Skills, from Publications Services, and from the Division 
of Supply and Property Management. Scoring costs and part of -the preparation 
and materials costs for the uniform^ portion were obtained from interviews of 
resource teachers and regular teachers. 

All of the costs for the other exams were obtained from interviews with 
resource teachers and"^ other teaching staff. These interviews were conciucted 
in nine schools in each subject— three at the junior high and six and the 
senior high levels. All 'three administrative areas were equally 
represented. 

The math teacher interviews focused on the experience ^of administering the 
Geometry B exam in June, 1982. Information was also collected on what was 
anticipated for Geometry A and Algebra' lA in January, 1983. The English 
interviews focused on new English exams iA Introduction to High School English 
and Narrative-Drama as Literature I, looking at bpth experience in development 
and expectations for administration in January, 1983. In ^these interviews 
teachers were also asked about departmental or teacher-made exams in other 
subjects. . _ 

All of the interviews were aimed primarily at determining the time spent by 
various staff members on tasks related to each type of exam. This time was 
transformed into dollars by using average salaries supplied by the Department 
of Personnel Services. All salary figures were increased by .a factor of 30 
percent to coyer overhead such as fringe benefits, equipment maintenance, 
electricity, etc. ^ 
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Computation of Costs 

The data obtained from each respondent were aggregated to produce average 
costs for each type pf test. This was done in two ways. Development and 
preparation costs were aetermined on a per examination basis on the assumption 
that they would not vary' as a function of the number of students enrolled in a 
course. Costs for materxais and scoring costs, on the other hand, were< 
adjusted for course enrolments as they are directly related to this factor. 
Presented in the following sections are brief summaries of how these averages 
were computed; 

Calculating Development and Preparation Costs 

Overall development and preparation costs of the countywide uniform 
portion were computed by totaling tnfiL,_s<I^^X • 1982 EYE costs for item 
writing workshops and one-fourth of tire salary for the subject 
coordinators in English and math. However, since staff produced two exams 
for each of several courses, the following algorithm was us^^d to develop 
cost estimates for a single test. First, total staff costs Uere divided 
by the number of courses covered, 9 for math and 16 for English. Second, 
the cost of a single exam was estimated by taking the per course average 
and discounting it by 30 percent. The per course average was not simply 
halved because it was felt that the development of a single exam is 
proportionally more costly than that of two. ' Start-up costs such as 
training item writers,, planning, and allocation of responsibilities 
contribute to this difference. 

For the other three types of exams, the procedures were slightly 
different, and it was necessary to make some adjustments in order to make 
the data comparable to that obtained for the countywide portion. 
Development and preparation costs were determined by using the average 
costs per exam provided by the teachers in the interview sample. It was 
asstimed that this average was representative of all schools in the 
county. Thus, to determine total county development and preparation 
costs, this average was multiplied by 35, the number of schools in the 
county having at least one grade between 9 and ?2. The cost per school 
for teacher-made exams was doubled on the assumption that there would be 
an average of -two teachers per course in a school if they were serving the 
same number of students as the courses included in the pilot program. 
Further, it was assumed that these teachers would develop their exams 
independent of each other. 

Calculating Materials and Scoring Costs 

Materials and scoring costs vere based on the average per student cost 
determined from* the teacher interviews. To compute this average, the 
total costs for all of the sample schools were determined in English and 
math. For the Other Departmental and Teacher-made Exams, these costs came 
from different courses in each school. The enrollments in these courses 
in each school were then totaled; The sample total cost for each type of 
exam was divided by the sample total enrollment for edch type of exam to 
get the averages for each subject area. To make . these data; more 
comparabia with those for the pilot' program, the averages were multiplied 
by the countywide enrollment in Geometry B (5,600) and Introduction to 
High School English or Narrative-Drama as Literature I (both 3,400). For 
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the countywide uniform exams, materials costs from the print shop ware 
added in. 



Calculating Cost Projections 



The computation of the different kinds of cost discussed above made it 
possible to project the total cost of each type of exam for a semester for 
all courses in English and- mathematics. The development and preparation 
costs for the countywide uniform exam for the projection for all courses 
were determined by multiplying the per exam average discussed above by 16 
in math and 30 in English. These numbers represent the number of courses 
offered countywide in each, of those areas in the fall of 1982. For the 
other exams, the development and preparation costs were derived by 
multiplying the costs for one exam by 10 because the average number of 
courses offered per school is 9.7 in math and 9.5 in English. Materials 
and scoring costs for this projection were determined by multiplying the 
per student cost by thenumber of students . enrolled in the 16 math courses 
-and 3© English courses in the fall of 1982. 
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